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Abstract 

The first obstacle in building a Geometric Quantization theory for nilpotent orbits of a 
real semisimple Lie group has been the lack of an invariant polarization. In order to gen- 
eralize the Fock space construction of the quantum mechanical oscillator, a polarization 
of the symplectic orbit invariant under the maximal compact subgroup is required. 

In this paper, we explain how such a polarization on the orbit arises naturally from 
the work of Kronheimer and Vergne. This occurs in the context of hyperkaehler geom- 
etry. The polarization is complex and in fact makes the orbit into a (positive) Kaehler 
manifold. We study the geometry of this Kaehler structure, the Vergne diffeomorphism, 
and the Hamiltonian functions giving the symmetry. We indicate how all this fits into a 
quantization program. 

1 Introduction 

Quantization is a procedure for constructing a quantum system with symmetry out of a 
classical system with symmetry. No axiomatic or even systematic method of quantization is 
known. Instead, quantization exists as an empirical science, made up of a growing series of 
examples. In many ways, quantization is an art. 

The nature of the classical and quantum systems under consideration depends on the 
context and on the scope of the investigation. At present a universal sort of quantization 
scheme seems completely out of reach. Such a scheme would have to include the quantization 
of gravity as well as the quantization of classical field theory. In fact, some physicists believe 
that even familiar classical theories must be modified in order that they can be "quantized" 
to give a consistent and meaningful quantum theory. 

There is a rather clear "beginning level" at which to formulate and study the quantization 
problem. This is the case where one starts with a classical Hamiltonian mechanical (dynam- 
ical) system with symmetry. Such a system is given by a phase space (M, w) together with a 
Hamiltonian function F and a Lie subalgebra 

C C^{M) (1.1) 
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Here (M, w) is a real symplectic manifold; i.e., a; is a symplectic form on a smooth manifold 
M of dimension 2n. The symplectic structure oj defines a Poisson bracket { , } on C°°(M), 
giving it the structure of a Poisson algebra. The Hamiltonian is a fixed smooth function 
F : M — > M which determines the time evolution (the dynamics) of the system. Each smooth 
function ^ on M determines a Hamiltonian vector field by the formula ^^Ju; + d0 = 0. 
The Poisson bracket is given by {0, "0} = '^^(V') = ^{C(f>i^tl>) a-^d satisfies 

[U^^i>]=^{M (1-2) 

Thus the Hamiltonian vector fields of the functions G give an infinitesimal Lie al- 
gebra action of g on M. This constitutes infinitesimal Hamiltonian symmetry. If the g-action 
integrates to a smooth action of a Lie group G on M, then this G-action is called Hamilto- 
nian. Regardless of integration, the inclusion g ^ C°°{M) defines a smooth infinitesimally 
g-equivariant moment map 

M^g* (L3) 



If G-acts transitively on M in a Hamiltonian fashion, then the moment map ( |1.3D is 
just a covering onto a coadjoint orbit of G. This focuses attention on coadjoint orbits as 
the "elementary Hamiltonian spaces" (up to covering). Moreover, each coadjoint orbit P 
has a canonical symplectic structure a, sometimes called the KKS (Kirillov-Kostant-Souriau) 
symplectic structure, derived from the Lie algebra bracket on g. Indeed, each x G g defines 
a linear function cj)^ on g* and hence on P. Then a is the unique symplectic form such that 
the mapping 

g ^ C~(P), (L4) 
preserves brackets, i.e., is a Lie algebra homomorphism. This discussion applies equally well 



in the category of holomorphic symplectic manifolds; cf. [BK1|. 

An outstanding problem is the quantization of conical coadjoint orbits Ok of a real 
semisimple Lie group Gr. Here we take Gk to be a real form of a connected and simply- 
connected complex semisimple Lie group G with Lie algebra g. We may identify giR = LieG^ 
with its dual g^, and then the conical orbits Or identify with the orbits of nilpotent elements 
in gR. These are the so-called "nilpotent orbits". The irreducible unitary representations 
arising from quantization of nilpotent orbits are often called "unipotent" representations. 
These are examples of "singular representations" . 

In order to start a quantization program for nilpotent orbits, we need at the outset an 
invariant polarization of Or. In analogy with the well-known quantization of the harmonic 
oscillator, we want a polarization invariant under a maximal compact subgroup of Gr. 
(In general Or does not admit GR-invariant polarizations.) 

Remarkably, a i^R-invariant polarization arises, in a uniform natural manner on every 
real nilpotent orbit, from the work of Kronheimer and Vergne. This comes about by first 
working on the complexification O of Cr; O C g is a complex nilpotent orbit of G. We let I 
denote the natural complex structure on O; then Or is an I-real form. 



Kronheimer (|Kr|) in 1990 identified each complex nilpotent orbit O as an instanton 
moduli space. In particular the holomorphic symplectic structure (I, S) on O extends to a 
hyperkaehler structure ((7, 1, J, K, wi, wj, wk) where = toj + iujk- We outline the Kron- 
heimer model of O in §^. 
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Then Vergne in 1995 used this to discover a diffeomorphism 

V : Or ^ y (1.5) 

of each real nilpotent orbit Or to a complex i^-homogeneous cone Y, where K C G is the 
complexification of K^. This recovered the Kostant-Sekiguchi (|S^) correspondence. 

The upshot of Vergne's work on the Kronheimer instanton model of O is that Or is a 
J-complex submanifold of O^. Moreover, Or is then a Kaehler submanifold of O with respect 
to (J,a;j). But then wj = ReS is just the real KKS symplectic form on Or. 

So the "new" complex structure J provides a complex polarization on Or and moreover 
makes Or into a Kaehler manifold which identifies with y as a complex manifold. We explain 
the Vergne theory and the Kaehler structure in §^ and §0. In §^ we also give a different proof 



of Vergne's result (see especially Proposition 3.f: and Corollaries 6.5 and 6 



In §0,^P we develop the properties of the Vergne diffeomorphism and the Kaehler struc- 



ture. Our first main result is the Triple Sum formula in in Theorem 7.9. This leads to our 
key result for quantization in Theorem |9.3| on the nature of the Hamiltonian functions (f)^ . 
Indeed we have the Cartan decomposition 0r = £r © pR. While the Hamiltonian flows of 
the functions (j)^, x £ Jr, preserve J, the flows of the remaining functions (j)'" , v p^, do not 
preserve J. The question then is how will they quantize. On a classical level, we can ask 
how to write down (p^ in terms of holomorphic and antiholomorphic functions. The answer 



in Theorem |9.3| is that (j)^ is the real part of a holomorphic function. The interpretation of 
this is discussed further in §|9[ 

In §P, we explain another aspect of the Kaehler structure, namely that there is a global 
Kaehler potential po on (Or, J,cr). This function po : Or — > M is KR-invariant and uniquely 
determined by the condition that it is homogeneous of degree 1 under the Euler scaling action 
of M+. 

This Kaehler potential arises by restriction from the hyper kaehler potential on O. In 
§^§ll|)§@ we develop the basic theory of hyperkaehler manifolds, hyperkaehler cones and hy- 



per kaehler potentials based on results from |HKLR] and 



The importance of the Kaehler potential po is this: in our quantization program for real 
nilpotent orbits, po plays the role of the Hamiltonian, i.e., the energy function. Moreover, 



Po gives rise in Theorem 8.6 and Corollary 8.7 to a realization of T*Y as a holomorphic 



symplectic complexification of Or. 

Our quantization program building on this geometry will be developed in subsequent 
papers. See also [ll|,@, pK2| . 



In the quantization of (Or, J,cr), we want to "quantize" the Hamiltonian functions (f>^, 
z G qr, by converting the (p^ into self-adjoint operators Q{(j)^) on a space of holomorphic half- 
forms on (Or, J) = Y. The conversion must satisfy in particular Dirac's axiom that Poisson 
bracket of functions goes over into the commutator of operators so that Q({0^, </>"'}) = 

A main idea coming out of Corollary ^.7| is that we can try to "promote" the functions (j)^ 
on Or to rational functions on the holomorphic symplectic complexification T*Y. For this to 
work, we need some sort of analyticity and algebraicity for the embedding of Or into T*Y. 

The appropriate notion combining analyticity and algebraicity here turns out to be that 
of a Nash embedding. In the Appendix, we give an outline of Nash geometry, starting from 
the theory of real algebraic varieties. O. Biquard has proven in [Bi] that the hyperkaehler 
potential on O, and hence the 5'0(3)-action on O and Vergne diffeomorphism (|1.5D, are Nash. 
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2 Hyperkaehler Manifolds 

In this section, we review and perhaps clarify some basic notions of hyperkaehler geometry 
that we use throughout this paper. 

A hyperkaehler manifold {X, g, Ji , J2, J3) is real manifold X of dimension 4n together with 
a Riemannian metric g and three complex structures Ji,J2,J3 such that (i) J1J2J3 = —1 and 
(ii) 5 is a Kaehler metric with respect to each of Ji,J2,J3. 

Then by (i), Ji,J2,J3 satisfy the quaternion relations 



Here a,b,c £ {1,2,3} are distinct and £abc = sgn(a6c). Thus every tangent space of X 
becomes a quaternionic vector space. 

By (ii), X has three Kaehler manifold structures (Ji,tJi), (J2, 0^2)1 (J35'^3)) all with 
Kaehler metric g. The Kaehler forms uJa are given by g{u,v) = uJa{u,Jav)- We call these 
Kaehler manifolds Xi, X2, X3, respectively. 

The data {X, g,uji, 0:2,^^3) serves equally well to define the hyperkaehler structure as we 
may recover the complex structures by the formula 



t2 _ t2 _ t2 
_ J2 — J3 



= -1 




LOc{u,v) = iOa{^bU,v)£abc 



(2.1) 



We define three complex 2-forms on X 



fti = UJ2 + iuJs, r22=W3 + 2a;i, il.^ = loi + iuj2 



(2.2) 



Then 0,a is Ja-holomorphic. This is shown in [ HKLR , pp. 549-550]. 



Inside the quaternion algebra 



M = M e Mi e Mj © Mk 



we have the standard 2-sphere 



{q = ai + bj + ck \ \q\ = 1} 



of pure imaginary quaternions of unit norm. 

Corresponding to a point g = ai + 6j -|- ck on S"^ 



we have the pair 



aJi -I- 6J2 + cJs 



and 



OJq = auji + buJ2 + CUJZ 



Then {X,g,iq,ojq) is again a Kaehler structure on X with complex structure Jg and Kaehler 
form OJq] we write Xq for this Kaehler manifold. Thus we have a 2-sphere Sx of Kaehler 
structures (Jg,cjg) on X and we have identified Sx with S"^. 
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Let g HH- rg be the standard rotation action of SO{3) on 5^. This induces an S'0(3)-action 
on Sx given by r • Jg = J^g and t ■ LOg = ujrq- Let 

Cq C 50(3) (2.3) 

be the circle subgroup of which fixes q ^ S"^. 

The generahzation of ( p.2| ) is that any q' € 5^ orthogonal to q determines a Jg-holomorphic 
symplectic form uOqi + iwg// on X where q" = q x q' is the cross product of q with q'. 



Example 2.1 The first example of a hyperkaehler manifold is the flat quaternionic vector 
space. Let X = M^" with standard linear coordinates where r = 0, 1, 2, 3 and s = 1, . . . , n. 
We may make X into an n-dimensional quaternionic vector space, where H acts by left 
multiplication, in the obvious way so that the functions 

= x° + + + x^k (2.4) 

are quaternionic linear coordinates. 

The following data deflnes a hyperkaehler structure on X: left multiplication by i, j and 
k give the complex structures Ji, J2, J3 so that 

where (abc) is a cyclic permutation of 1,2,3. Under J a, X identifles with C^" with linear 
holomorphic coordinate functions + ix^, x^ + iXg. The hyperkaehler metric is 

g = ^{dx:f (2.6) 

r,s 

The three Kaehler forms loi,uj2, ^3 are 

n 

uJa = Y^ dx° A dx^ + dxl A dx^ (2.7) 

s=l 

Next we introduce hyperkaehler symmetry into the picture. Let U he a Lie group. A 
hyperkaehler action of U on {X, g,Jq,uJq) is a smooth Lie group action of C/ on X which 
preserves all the hyperkaehler structure. 

From now on we assume that [/ is a compact connected semisimple Lie group and we 
have a hyperkaehler action of C/ on X. Then differentiation gives an inflnitesimal action of 
the Lie algebra u of [/ by the vector fields where = ^{^^oi^W —tu) ■ p at p G X. In 
other words, we get a Lie algebra homomorphism 

u^53ectX, u^e" (2.8) 

Now consider each Kaehler manifold Xq. We let C°°{X)i^^ denote the algebra C°°{X) 
equipped with the Poisson bracket defined by iOq. 

The [/-action on X is symplectic with respect to ujq and consequently, since u is semisim- 
ple, is Hamiltonian. This means that we can solve the equations -\iOq+dCq = for functions 
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Cq such that {CqXq} = Cg"'^^ • The momentum functions Q are uniquely determined. So we 
get a Lie algebra homomorphism 

u^C~(X)^,, u^Q (2.9) 

The corresponding [/-invariant moment map 

Cg-.X^u (2.10) 

is defined by (qip) = {u, Cqip))u- Here we identify u ~ u* by means of the Killing form ( , )u. 

Consider now the three moment maps (i = d, C2 = Cj; Cs = Ck, Putting these together 
we obtain a triple moment map 

C = (iCi,jC2,kC3) :X^iuejueku (2.11) 

Let G be the complexification of U. Then G is the complex semisimple algebraic group 
characterized by either of the following properties: (i) any linear representation of [/ on a 
complex (finite-dimensional) vector space extends uniquely to a linear representation of G, 
or (ii) [/ is a compact real form of G. It follows from (ii) that U and G have the same 
fundamental group. 

We assume now that U, and hence G, is simply-connected. The Lie algebra of G is the 
complex semisimple Lie algebra 

g = u(g)C = ueiu (2.12) 

We identify 0^0* using the complex Killing form ( , )g on 0. We note that {u,v)u = iu,v)g 
for u,v £ u. This follows because an M-linear map L : u — > u determines a C-linear map 
: ^ and then Tr^L = Tr^Lc- 

Now we consider the holomorphic symplectic manifolds {X,ia^^a)^ o, = 1,2,3. We let 
R^"^{Xa) denote the algebra of Ja-holomorphic functions on X equipped with the Poisson 
bracket defined by Via- 

Since C^ui^ = Ja, w G u, it follows that is the real part of a unique Ja-holomorphic 
vector field on X. Precisely, = ReH^ where 

K = \{e-i^an (2.13) 

Then we get the bracket relations for n, w S u 

Now we have an infinitesimal Ja-holomorphic Lie algebra action 

^ ^tci,^.Hoi{X), z = u + iv^'Bl = 'S:, + iEl (2.14) 



of on Xa- This is the complexification of the infinitesimal u-action (|2.8| ). If ( ^.13 ) integrates 



to a holomorphic G-action on (X, J^), then we will say that the [/-action on X complexifies 
with respect to Ja. 
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Regardless of integration, the infinitesimal action ( p. 14 ) of g preserves the holomorphic 



symplectic form 0^ defined in ( |2.2D . Then, since q is semisimple, the infinitesimal action 
( p.l4 ) is Hamiltonian. We have a unique complex Lie algebra homomorphism 



^ z ^ K (2.15) 
given by momentum functions so that j fi^ + = 0. Then 

^u+iv = ^u+.^v ^n = ^u^i(;u (2.16) 

where (abc) is a cyclic permutation of 1,2,3. 

The corresponding g-equivariant J^-holomorphic moment map 

<^a-Xa^Q (2.17) 

is defined by ^aip) — (^i ^aip))s- Thus we get the three maps 

^l=C2 + <3, $2 = C3 + <1, $3 = Cl+<2 (2.18) 



The formulas ( p.l6D -( p.l8 ) encode a lot of information about the coupling of the complex 



and symplectic structures on X, as $a is Ja-holomorphic. In particular they show how the 
real functions Cf, ^2 ) Cs give rise to holomorphic functions on X. 



Example 2.2 We continue the discussion of X = H" from Example |2.1| . Let U be the group 
of all R-linear transformations of X which preserve g and commute with the EI*-action on X. 
Then U is the familiar model of the compact symplectic group Sp{n). Clearly this [/-action 
preserves all the hyperkaehler data on H". In the case n = 1 then U ~ SU{2) and moreover 
U acts by right multiplication by quaternions of unit norm. 

The [/-action complexifies, with respect to any complex structure Jg G Sx, to a complex 
linear complex algebraic action of G ~ Sp{2n, C) on H". This action is transitive on H" — {0}. 

We have a free Z2-action on H"' — {0} by multiplication by ±1. This Z2-action preserves all 
the hyperkaehler data on H" — {0} and commutes with the SU{2) and G-actions. The quotient 
O = (H" — {0})/Z2 inherits a hyperkaehler structure, an action of SU (2) /7j2 — S0{3) and a 
[/-action. 



3 Hyperkaehler Cones 

In this section we explain the notion of a hyperkaehler cone. 

To begin with we recall that a symplectic cone of positive integer weight /c is a symplectic 
manifold (M, cu) together with a smooth action 

M+ X M M, (t, m) ^ jt{m) 

of the group of positive real numbers such that 

jtuj = t^uo (3.1) 

This means that the R+-action scales the symplectic form and it has weight k. 
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The prototype example is the case where k = 1 and M = T*Q is a cotangent bundle with 
its canonical symplectic structure. Here M"*" acts on T*Q by the linear scaling action on the 
fibers of the projection T*Q Q. 

Let r] be the infinitesimal generator of the ]R"''-action. Then differentiating (3.1) we get 
the equivalent condition 

CfjUJ = ku! (3.2) 
It follows that oj is exact with symplectic potential ^{ri_\uj); i.e., 

CO = d(— r/_iu;) 
k 

We conclude in particular that a symplectic cone is non-compact (and has positive dimension). 

Next we define a Kaehler cone of weight A; to be a Kaehler manifold {Z,J,uj,g) together 
with a smooth action 

7 : C* X Z ^ Z, {s,m)^ 7^(m) (3.3) 
which satisfies the three conditions 

(i) the action 7 is holomorphic, {ii)^*uj = \s\^uj, {iii)'j*g = \s\'^g (3.4) 

(These are consistent with redundancy). 

The condition (i) means that the map (|3.3| ) is holomorphic. So (i) implies 7* J = J. Also 
any two of 7*J = J, (ii), (iii) imply the other. 

To work out (ii) and (iii), we use the product decomposition 

C* = M+ X 

So the C* action splits into a product of an M"'"-action with an 5'^-action. Then (ii) and (iii) 
say: uj and g are homogeneous of degree k under the ]R"''-action, but they are fixed by the 
5^-action. 

Thus 5*^ acts by Kaehler automorphisms. In particular, the S'^-action is symplectic and 
so has a moment map on Z at least locally with values in M. We can write this moment map 
as |/?. Then p is a local Kaehler potential, i.e., p satisfies 

iddp = u) (3.5) 

where d = d + d \s the standard decomposition of d into (1, 0) and (0, 1) parts. 

The Lie algebra of C* is C = M © Mi with [1, i] = 0. Differentiating the C*-action we get 
an infinitesimal vector field action 



at 



7cxp-to (3.6) 



We put 

ri = ip-^ and = V"' (3.7) 



Instantons and Kaehler Geometry of Nilpotent Orbits 



9 



so that r] and 6 are, respectively, the infinitesimal generators of the actions of and S^. 
Since "0 is a (real) Lie algebra homomorphism we have 

[V,e]=0 (3.8) 

Notice that the infinitesimal generator of the holomorphic action of C* on Z is the holo- 
morphic vector field 

E = \v- \iO (3.9) 



Now, we can give an equivalent infinitesimal version of the conditions ( |3.4| ) on 7: il) must be 
C-linear, i.e., 

e = iri (3.10) 

and also 

>CryJ = 0, Cr,UJ = kuJ, Cf^g = kg, Cg J = CgUJ = Cg g = (3.11) 

Notice that the condition Cj^J = itself implies [r], Jrj] = 0. 

Now we define a hyperkaehler cone of weight /c to be a hyperkaehler manifold {X, g, Jq,ujq) 
together with a left EI*-action 

7 : M* X X ^ X, {h,m)^ jh{m) (3.12) 

which satisfies the three conditions 

(i) 7^Jg = -^h-^qh the action of C* on X is J^-holomorphic 

{ii) -flug = \h\^ujh-igh (3.13) 
(m) 7*5 = \h\''g 

Here heM.*,qeS^ and 

C*g = {a + bq\ (a, b) G M^, (a, b) / (0, 0)} (3.14) 

Again (i)-(iii) are consistent with redundancies. It suffices to check (i)-(iii) just for q = i,j,k. 
We have the natural direct product decomposition 

M* = M+ X SU{2) 

The formulation of (i)-(iii) in terms of the component actions of M"*" and SU{2) is: (a) for all 
q, acts Jg-holomorphically and scales g and cOg so that they have weight k, (b) the action 
of the circle 

= {cost + qsmt\t e M} C SU{2) (3.15) 

on X is Jg-holomorphic, and (c) SU{2) acts isometrically and permutes the Kaehler structures 
Xq according to the standard action of SU{2) = 5*0(3) on S^. 
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We can also rewrite the conditions (i)-(iii) at the infinitesimal level. Indeed the Lie algebra 
of the multiplicative group M* is IHI with Lie bracket [u, v] = uv — vu. We have a standard 
basis jo, ji, j2, ja of HI with 

jo = 1, ji = i, j2=j, js = k 

and bracket relations 

[jo,ja] = and [ja,jb] = 2eaf,cjc (3.16) 
Differentiating the EI*-action we get an infinitesimal vector field action 

: M ^ QJectX, g ^ (3.17) 

We put 

r? = V"^ and 6'a = V"""", a = 1,2,3 (3.18) 

Then r] is the infinitesimal generator of the M+-action and 9a is the infinitesimal generator of 
the Tj-action. The bracket relations are 

[r/,^,] = o, [ea,eb] = -2eabcec (3.19) 

Now we can give an equivalent version of the conditions ( |3.13| ) on 7: ip is H- linear, i.e., 

9a = 3aV, a = 1,2,3 (3.20) 

and also 

C^iVa = kuja, CrfJa = 0, €,^9 = kg ^2 

In (|3.2l| ), we do not require a,b,c distinct, but instead we define 

J sgn{abc) if a, b, c are distinct 
£abc "j^ Q otherwise 

We call a vector field action (3.1?) satisfying (3.18)-( p.21D infinitesimally conical with weight 
k. 

Lemma 3.1 Any infinitesimally conical vector field action ofM on X necessarily has weight 
k = 2. 

Proof If a, b, c are distinct then (|3^ ) and (|1|) give 

9a-iu;b = -Sabcll-i^c (3.22) 

But then 

-2eabc^c = Ce^UJb = d{6a-iUJb) = -^abAijl -^^c) = -£abcJ--v^c = -kSabc^c 

Hence k = 2. □ 

So any hyperkaehler cone necessarily has weight 2. From now on, we assume A; = 2 in 
( p3D and (|filj). 
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4 The Hyperkaehler Potential 

In this section, we explain, on the global level, the relation between the hyperkaehler cone 
structure and the hyperkaehler potential. This was worked out locally in |Sw|; see also 
| HKLR] , pg. 553] for part of this. 



Corresponding to each complex structure Jg on X , we have the decomposition d = dq+dq 
of the exterior derivative into (1, 0) and (0, 1) parts. We put 



d^ = -\iqd = -\{dq-dq) (4.1) 



A global hyperkaehler potential on X is a smooth function p : X ^ M which is a simultaneous 
Kaehler potential for each Kaehler structure Xq, i.e.. 



idqdqp = M'qP (4.2) 



for all q ^ S"^. It follows easily that p is a hyperkaehler potential iff p is a Kaehler potential 
for Xi , X2 and X3 . 

Proposition 4.1 Suppose {X,Jq,uJq,g) admits a global hyperkaehler potential p : X ^ M. 
Let T] be the vector field on X defined by 

r]_ig = dp (4.3) 

and set 9q = Jqrj. 

Then p, after perhaps being modified by adding a constant, satisfies rjp = 2p so that p is 
homogeneous of weight 2. The vector fields r], 61,62, 63 define an infinitesimally conical vector 
field action ofM on X where 61,62,63 define the infinitesimal 50 [3) -action. 

The potential p is SO (3) -invariant. For each q & S'^ , the Hamiltonian flow of p with 
respect to tUq integrates the vector field 6q. In other words, we have 

6q^LUq + dp=0 (4.4) 
so that p is a simultaneous moment map for each infinitesimal S^ -action defined by 6q. 
Proof To begin with, we observe that if a vector field rj and a function p satisfy ( |4.3| ) then 
p is a hyperkaehler potential -4=^ jCrjUJa = 2cja for a = 1,2,3 (4-5) 
Indeed the computation 

iV-iUJa, = ^aiV, = ai^ar], = -5'(^, JaC) = ( " dp, JaO = { - Jadp, = (2d^p, ^ 

gives rj-iu>a = 2d^p. Applying d to this we get 

Cr^i^a = 2dd^p 



This implies ( |4.5| ). 

Now suppose p is a hyperkaehler potential. We need to prove all the relations in ( 3.2l| ); 
in fact we can ignore the two involving the metric g as they are redundant. The relations 
CrjOJa = 2u;a are done in ([4.51). These imply the relations C^3b = because we can apply Cr^ 
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t o (|2.1| ). This in turn gives [r],Oa] = Crj{JaV) = 0. To prove the second hne of relations in 
( p.21[) we first observe that g{u,v) = uJa{u,3av) = iOa{—JaU,v) gives 



-9a-iUJa =V-i 9 



(4.6) 



So 



^ea^a = diOa-iuJa) = -d(??_ig) = d^p = 



For a,b,c distinct we find using ( 3.22 ) 

Cea^b = d{ea-iuJb) = -eabcd{r]-\uJc) 



Next, applying Cq^ to ( |2.1D we find that C^^Ja = —2eabcJb- This proves the six independent 
relations in ( p. 211) . Also 

[0a, (^b] = ^Oa^b = ^Oai^bV) = — 2eabcJc'? = —'^^abcdc 

Thus we have an infinitesimally conical H-action. 

Now applying £^ to ( p^ ) we find £^(dp) = 2dp. So -qdp = 2dp and therefore rjp = 2p + C 
where C is constant. We can replace p by p + C/2 so that r]p = 2p. Next applying Cg^ 
to (|4.3|) we find C0^{dp) = 0. So Ce^p = Ca where Ca is constant. But then it follows 
from the semisimplicity of su(2), in particular from the relations [6a,0b\ = ~2£abc&c, that 
Ci = C2 = Cs = 0. Hence OaP = 0. 

Finally, it suffices to check ( 4^ ) for any single g € 5^ because of the 5o(3)-action. Clearly 
Oa-iiOa + dp = follows by IK3) and (O). □ 



We may think of a hyperkaehler IH*-conical structure on X as a family of Kaehler cone 
structures on X which satisfy additional properties. Indeed, for each q £ 5^, Xg is a Kaehler 
cone with respect to the action of C* = M"*" x Cg where the action of Cg integrates 9g. 



A converse to Proposition 4.1 follows easily from the proof. 



Corollary 4.2 Suppose H^^j^f^^^{X) = and X admits an infinitesimally conical vector 
field action ofM. Then (4.3) has a smooth solution p on X (unique up to the addition of a 
constant function), and p is a global hyperkaehler potential. The further condition rjp = 2p 
uniquely determines p. 



Proof It is enough because of (|4.5|) and the relation CrfUJa = 2u)a in ( 3.21J ) to produce a 
solution /) to (O). Since ii\eRharrX-^^ ~ 0) the problem reduces to showing that the 1-form 
r/_ig is closed. This is easy: the general fact (4.6) and one of the relations in ( p. 21 ) give 
d(r/_i g) = -Ce^uJa = 0. □ 



Example 4.3 We continue discussing X 
plication action of H* on X given by 



from Examples 2.1 and 2.2. The left multi- 



h o (gi 



{hqi, hqn) 
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makes X into a hyperkaehler cone of weight 2. The natural 50(3)-action on the 2-sphere 
Sx of Kaehler structures is induced by the S'C/(2)-action on X defined by left multiplica- 
tion by quaternions of unit norm. The infinitesimal generator r/ of the M^-action and the 
corresponding hyperkaehler potential p are 

r,s * r,s s 

The vector fields 6a, a = 1, 2, 3, are 

where {abc) is cyclic. 

Let M*'" = iM e jM © kR. We have a weight-2 left action of M* on M*"" defined by 

h*w = \h\'^{hwh~^) (4.9) 

This is just the product of the degree 2 scaling action of with the spin 1 action of SU{2). 
We then get a tensor product action of H* x [/ on 

iueju©ku = IH^™(g)u (4.10) 

given by, for w € and u G u, 

{h, a) ■ (wu) = (/i • w)AdaU (4-11) 



Lemma 4.4 Suppose a hyperkaehler cone X has a hyperkaehler action of U which commutes 
with the M.* -action. Then the triple moment map ( p.ll] ) is equivariant under M* x U with 
respect to the action (4.11). 



Proof This follows immediately by transforming the equation ^" _i + d^^ = under the 
]HI*-action. □ 



Lemma 4.5 Let Z be a complex manifold with Hj-^^^^^^^Z = 0. Then 

(i) A smooth function p : Z ^ M is pluriharmonic {i.e., ddp = 0) if and only if p is the real 
part of a holomorphic function on Z. Suppose further a compact group H acts holomorphically 
on Z and this action complexifies to a transitive action of the complexified group Hq on Z. 
Then 

(ii) The only H -invariant pluriharmonic smooth functions p : X — > M are the constants. 

(iii) If ujz is a Kaehler form on Z , then an H -invariant Kaehler potential on Z, if it exists, 
is unique up to addition of a constant. 
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Proof (i) This follows by a standard argument (see e.g. [ Kra| ). Indeed the 1-form i{d—d){p) 
is closed and so exact since -f^oeijham-^ ~ solve i{d — d){p) = dcf) globally for 

a smooth real-valued function cj). Then idp = dcj) and idp = id(p. But then f = p — i(p 
satisfies df = which means / is holomorphic. (ii) By (i), we have p = Ke f for some 
holomorphic function /. We see easily that / is i7-invariant. But then / is iJ£-invariant. 
Now the transitivity assumption forces /, and hence p, to be constant, (iii) follows from (ii) 
since the difference of any two potentials is pluriharmonic. □ 

An immediate consequence is 



Proposition 4.6 Suppose X is as in Proposition 4.1 and X carries a hyperkaehler action of 
U which commutes with the infinitesimal M-action. Then the homogeneous degree 2 solution 
p to ( ^^ ) is, up to addition of a constant, the unique U -invariant hyperkaehler potential on 
X. 



5 Instantons and the Kronheimer Model of a Complex Nilpo- 
tent Orbit 



In this section we recall some main results from [Ki]. First we construct the instanton space 
A4(k) , and then we explain how A4{k,) is isomorphic to a complex nilpotent orbit O. 

The subspace H*™ of pure imaginary quaternions is a Lie subalgebra of H isomorphic to 
so(3); cf. ( 3.16| ). From now on, we may identify so(3) with H*"^. The adjoint action of S0{3) 
identifies with the standard spin 1 action of S0{3) on H*™', which we write as (r, w) i— > r * it;. 

Let -L(so(3),u) be the space of M-linear maps 

^:5o(3)^u, w ^ Au, (5.1) 

An element A{t) of the path space V = C°°(M, L(5o(3), u)) is given by a triple 

Ait) = {Aiit),A2it),Asit)) 

where Aa{t) = A^j^{t). The adjoint actions of SO{3) and U define a representation of 
SO{3) X U on L(so(3),u). This gives a representation of S0{3) x U onV defined by 

(r, u) ■ A,^{t) = Adu ■ A,-i,^{t) (5.2) 

Let Ai C V he the subspace of paths A{t) which satisfy the system of three differential 
equations 

^^Aa = -2Aa-[Ab,A,] (5.3) 

where (abc) is a cyclic permutation of 1, 2, 3. Now let 

k:so{3)^u (5.4) 
be a non-zero Lie algebra homomorphism; then /t is 1-to-l. We put 



Ca — '^( ja)) 



a = 1,2,3 
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Let C be the subspace of paths satisfying the boundary conditions 

hm A{t) = 0, Mm A{t) G C{k) (5.5) 

where C(k) C L(so(3),u) is the space of Lie algebra homomorphisms which are [/-conjugate 
to K. The action of S0{3) x U on V leaves stable A4 and A4{k). Thus we have a smooth 
action 

S0{3) X U ^ BiS M{k) (5.6) 

An instanton is an element of A4{k). We often write an instanton A{t) simply as 
with the time dependence being understood. 

It follows from the equations (|5.3| ) (see |Ki| ) that we have three well-defined [/-equivariant 
mappings 

Ci,C2,C3 :>1(^) (5.7) 

given by 

UA) = I lim e^'Aait) (5.8) 
We can arrange these three maps Ci) C2; Cs iiito the single map 

C = (iCi, jC2, kCs) : M[k) ^ iu e ju e ku (5.9) 



We have a natural (tensor product) action of 50(3) xU on iu © ju © ku because of ( f4.10| ). 
We see easily that C is (50(3) x C/)-equivariant. 

Each triple d = (^1,^2,^3) lying in C{k) gives rise to the "model" instanton 

D{t) = {l + e^')-\dud2,d^) 

Then UD{t)) = \da. 

It is easy to check that we have an action of on A^(k) given by 

\oA{t)=A{t-]^\og\) (5.10) 

This commutes with the action of 5*0(3) x U . 

The complexification of k is a complex Lie algebra embedding 

Kc : 50(3, C) Q 

The set M of all nilpotent elements in so(3,C) is 

A/" = {ai + 6j + ck I + 6^ + = 0} 

and TV — {0} is a single orbit under the adjoint action of 5*0(3, C). It follows that all the 
elements k{z), z ^ J\f — {0}, lie in a single adjoint orbit O in g. Then O consists of nilpotent 
elements in g; i.e., O is a complex nilpotent orbit. 
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In particular then 

= G-{e2+ ies) = G • (eg + iei) = G ■ (ei + iea) (5.11) 

Then O inherits a complex structure I from the natural embedding of O into g; we call this 
embedding 

$1 : O ^ (5.12) 

Or, equivalently, I is induced by the G-action. I and the G-invariant KKS holomorphic 
symplectic form S make O into a holomorphic symplectic manifold 

(0,I,S) (5.13) 

We recall that T, is the unique holomorphic symplectic form on O such that the adjoint 
action of G on O is holomorphic Hamiltonian with moment map $i. In terms of the holo- 
morphic component function <I>|, z S g, defined by ^f{w) = {z,w)q, this means that the 
S-Hamiltonian flow of the functions gives the G-action and the map 

Q^Ri^hoiiO), z^<^f (5.14) 

is a complex Lie algebra homomorphism with respect to the Poisson bracket on Ri^hoi{0) 
defined by S. 

The space A1(k) has a natural [/-invariant hyperkaehler structure {g, Ji, J2, J3, wi, a;2, 0^3); 
see |Kr, Remark 2, pg 476] and Q]. Kronheimer discovered 



Theorem 5.1 [Kt] 

(i) The map C i'n (5.9) is an (5*0(3) x U)-equivariant smooth embedding of manifolds. 

(ii) The three maps Ci7C2 7C3 ^^'"c moment maps for the U -action with respect to the three 
Kaehler forms , cj2 , on M.{k). 

(iii) For a = 1,2,3, the holomorphic moment map : A4{k) g given by is 1-to-l 
and has image equal to O. Thus we get a G-equivariant holomorphic symplectic isomorphism 

■■ M{k) O (5.15) 

from (A4(«;), Ja, r^a) to (0,I,I1). Here G acts on M.{k) by the Ja-complexification of the 
U -action. 

(iv) The SO{3)-action 

S0{3) ^DiSM{K) (5.16) 

preserves the Riemannian metric g and induces the standard transitive action of SO (3) on 
the 2-sphere of Kaehler structures on A4{k). 

In fact, (ii) determines uniquely the [/-invariant hyperkaehler structure on A4{k.). Let 
Ca = Cj^ . We further note 

Corollary 5.2 
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(i) The map is -equivariant with respect to ( 5.10| ) and the Euler scaling action on iu © 
ju © ku, i.e., 

C(AoA) = AC(^) (5.17) 

(ii) The isomorphism intertwines the product action o/M"*' x Ca on M{n) with the Euler 
scaling action of C* on O. 

(iii) The SO{3)-action on A4{k) is free. 

Proof (i) and (ii) are routine to verify. The Euler C*-action on O has only trivial isotropy 
groups, and so (ii) implies that the action of Ca, and hence of all of S0{3), has only trivial 
isotropy groups. A compact group action with only trivial isotropy groups is necessarily free. 

□ 

Next we define a left action of H* on by 

h ■ = \h\^ o Ah-i^h (5.18) 



This is the product of the square of the M"'"-action (5.10) with the 5'f7(2)-action defined by 
the S'0(3)-action on A4{k,) the spin homomorphism 

SU{2) SO{3) (5.19) 



We also have an E[*-action on iu © ju © ku, {h,p) ^ h • p defined by ([4. 91) . 
Let r/ be the infinitesimal generator of the square of the action (|5.1[1|). 



Corollary 5.3 The left M* -action on M.{k) defined in ( ^.18 ) satisfies 



Q{h ■ A) = h • C{A) (5.20) 

commutes with the action of U , and gives M.{k) the structure of a hyperkaehler cone. 

Proof The EI*-action on M.{k) clearly commutes with the ?7-action and satisfies ( ^.g| ) be- 
cause of the S'0(3)-equivariance in Theorem ^[^(i) and the R^-equivariance in Corollary 
P(i). 

Next we check the hyperkaehler cone axioms ( p. 13 ). By Theorem 5.1, the 5C/(2)-action 



transforms the tensors Jg, ojq, g according to ( 3.13| ). The action of the subgroup C* = C?^ of 



H* identifies (because of Corollary 5.2(ii)) with the square of the Euler C*-action on O under 
$a- Hence the action of C* is Ja-holomorphic. Furthermore, this implies that the action of 
the M"*" subgroup of H* preserves Ji, J2, J3, and transforms 01,02,^^3, and hence a;i,a;2,u;3, 
by the degree 2 character. Thus the M"'"-action also satisfies the axioms 

(HI)- ° 

Let rj be the infinitesimal generator of the action on M{k) of the ^'''-subgroup of H*, 
i.e., of the square of the action ( 5.10| ). We will say a function / on is homogeneous of 



degree r if this R^-action transforms / by the degree r character, i.e., ry/ = rf. 

Corollary 5.4 The hyperkaehler manifold JV[[k) admits aU -invariant hyperkaehler potential 
p, unique up to addition of a constant. The further condition that p is homogeneous of degree 
2 determines p uniquely. 
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Proof This follows by Corollary 5.3 and Proposition 4.6 as soon as we check that 



But Ai{Hi) is diffeomorphic to O by Theorem 5^(iii) and we have the well-known fact 



Lemma 5.5 The fundamental group of a complex nilpotent orbit O is finite. Consequently 

^DeRharn^ — ^■ 

This finishes the proof of Corollary □ 



6 Complex Conjugation on the Instanton Space A1(«:) 

We will introduce a family of complex conjugation maps on 

We start with the non-trivial Lie algebra involution q of so (3) given by 

^(aji + ^2 + cja) = -aji + 6j2 - cja (6.1) 

(The exact choice is immaterial, as any two choices are SO(3)-conjugate.) We assume now 
that the induced involution on k(so(3)) extends to a Lie algebra involution i? of u. Then i? 
determines a splitting 

u = aeb (6.2) 

where o is the 1-eigenspace of ?? and b is the (— l)-eigenspace of 'd. This gives the bracket 
relations 

[o,a]ca, [a,b]cb, [b,b]ca (6.3) 

Lemma 6.1 The involution 'dofu induces a hyperkaehler involution Q of which com- 

mutes with the SO{3)-action. 



Proof It follows from the instanton differential equations (|5.3| ) and the boundary conditions 
( ^.5| ) that any Lie algebra automorphism of u induces a hyperkaehler diffeomorphism of A4{k). 
Clearly Q commutes with the 50(3)-action. □ 

Next we extend ■(? in a C-antilinear manner to get a real Lie algebra involution i/ of g. We 
have 

= uem = aebezaezb (6.4) 

and so for x, G a and y, y' G b we get 

h'{x + y + ix + iy') = x — y — ix' + iy' (6.5) 

The point is that is a complex conjugation map on the Lie algebra g. The corresponding 
real form of g consisting of z/-fixed vectors is 

0M = aezb (6.6) 
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Now 0K is a non-compact real form of q, unlike u which is a compact real form. Indeed the 
Killing form of g, being negative-definite on u, is clearly indefinite on gjj. From now on, we 
call v "complex conjugation" and we put z = i>{z) for z G g. 

Clearly is compatible with k, in the sense that k(5o(3)) is complex conjugation stable 
in u; in fact K(<f(w)) = {}{Hi{w)) for w £ so(3). Hence complex conjugation preserves O inside 
g. We conclude 

Lemma 6.2 Our map z/ : g — s- g induces an antiholomorphic dijjeomorphism of O, which we 
again call v. So v : O ^ O is a complex conjugation map. 

See §0 and (|7.7| ) for how this arises in practice. We work with ( |6.6| ) now because the 
essential symmetry of the picture is more apparent this way (just as for the Sekiguchi corre- 
spondence). 

For a = 1,2,3, we call pull back complex conjugation on O through the holomorphic 
isomorphism in ( ^.15 ) to get an involution 



^,:M{k)^M{k), 7a = ^> (6.7) 

So ^ailaA) = z/($a^)- Then 7a is Ja-antiholomorphic and so 7a defines a complex conjuga- 
tion map on A1(k) with respect to Ja- 

Our aim is to figure out how the hyper kaehler data transforms under the involutions 7a. 
To this end, we find a formula for 7a in terms of Q and the S'0(3)-action on A4{k,). In 



particular, the rotation in H*™ about the ja-axis defines via ( ^.161) an isometric involution 

Ra : M{k) M{k) 

Proposition 6.3 Let (abc) be a cyclic permutation of 1,2,3. Then 

7a = QRb (6.8) 

Proof Let -i? : g ^ g be the involution defined by C-linearly extending : u — > u. Then 
induces a biholomorphic diffeomorphism of O, which we again call i?. It is easy to see that 

e = (6.9) 

The composition j3 = wd is the C-antilinear involution /3 : g — > g with fixed space u, i.e, /3 is 
complex conjugation with respect to u. So /? induces an antiholomorphic involution /3 of O. 
We now see that (|6.8|) holds if and only if 

Rb = KP (6.10) 

To compute $a/^, we first write out the map <I>a according to ( 2.16|) : 



^'a = a + <c (6.11) 

It follows that the diffeomorphism /? = of the instanton space M{k) satisfies 

P*a = Cb and P*Cc = -Cc 
But the rotation Rh transforms Cb and Cc in exactly the same way. This proves ( 6.10| ). □ 
Our formula (|6.8D immediately implies, because of Theorem |5.1|(iv): 
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Corollary 6.4 Let (abc) be a cyclic permutation of 1,2,3. Then the involution ja is an 
isometry of M.{n) which preserves {Jb^oj^) but negates (Ja,^;^) and (Jc,Wc)- Furthermore, 
commutes with the action ofCf,. 

In particular 7^ is a complex conjugation map with respect to J a which is also g-isometric, 
Jfy-holomorphic, LV^-symplectic and C^-invariant. 

Let M{ny°' be the subset of M{n) fixed by 7^. 

Corollary 6.5 

(i) ^4{Ky^ is a smooth submanifold of M.{k) with finitely many connected components, and 
each component has half the dimension ofM{n). 

(ii) is a symplectic real form of M{k) with respect to (Ji,r2i). 

(iii) A4{Ky^ is a Kaehler submanifold of A4{k) with respect to {g,J2,uj2). 

(iv) A4{Ky^ is a complex Lagrangian submanifold of with respect to (J2,n2). 

Proof If a : M ^ M is a finite-order automorphism of a smooth manifold, then the fixed 
space M° is a smooth submanifold. The tangent space of at m is the subspace of a-fixed 
vectors in the tangent space of M at m. 

Now Corollary |6.4| implies that is a Kaehler submanifold with respect to {J2,uj2)- 

So A4{Ky^ is a J2-complex submanifold and the restriction of uj2 to it is a real symplectic 
form vr. Furthermore A4{k)'^^ is a real form with respect to Ji and J3 and uJi and UJ3 vanish 
on A4{Ky^ . So A4{k)'^^ is Lagrangian with respect to = W3 + icoi and l^i restricts to vr. 
This proves (ii)-(iv). 

Finally we complete the proof of (i). First, A4{n)'^^ , being a real form of has half 

the dimension. Next, M{k)'^^ has only finitely many components since 71 = by (6.4.1), 
and so 

<^i{M{Ky') = O n (6.12) 
But O n 0K is a real algebraic variety and so has only finitely many components. □ 



Corollary 6.5 above and the next result below encode Vergne's result in "hyperkaehler 
language" . 

Proposition 6.6 We have 

$i(7W(k)^^) = On(oeib) 

MMi^V') = O nib (Bib) (6.13) 
MMi^V') = On(b©m) 

We already observed in ( |6.12[) that the first equality is clear, but the others reveal a subtle 
structure. 

Proof We start by computing how the functions C^, u € u, transform under 7 = 71. For 
any function / on A4{k), we put /'^ = 7*/ = / o 7. 

By (P), we have 7 = 7i = &Rb. So 7* = RIQ*. We find G*Ca" = Cf" and RIQ = taQ 
where ti = t^ = —1 and t2 = 1. Therefore, writing u = x + y where x € a and y € b, we get 
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Let u' = —X + y and u" = x — y. Then ( |6.14| ) says 

Ci(7^) = CiiA)', C2hA) = C2{A)", CsilA) = C3iAy, (6.15) 

Now we can compute the involutions ab, b = 1,2, 3, of O defined by $*aa = 71. Of course 
by the definition Q of 7i- Using (l2l6|) and we find 

<i>2iA)=UA)+iCiiA)^^2 hA) =C^{Ay + iC,{Ay 

6.16 

^3{A) = CiiA) + <2(A) ^ ^3(7^) = Ci(^)' + <2(^)" 

Consequently, the involutions 02 , 03 of O are induced by the involutions 02 , 03 of g given 
by, for xi,X2 G and 2/1,2/2 G i>. 



a2{xi +yi + 1X2 + %2) = -xi +yi- 1x2 + iy2 
asixi +yi + 1x2 + %2) = -xi +yi + 1x2 - iy2 



(6.17) 



So the fixed spaces are 

g"i=aeib, 0"2^beib, 3"^ = b e (6.18) 
For b = 1,2, 3, we have 

^b{M{Ky) = 0'"^ = Of^Q'"^ (6.19) 

This completes the proof. □ 
We set 

pi = On(aeib), p2 = On(beib), P^ = On(beia) (6.20) 
Now combining Proposition |6.6| with Corollary |6.5|, we get 



Corollary 6.7 For a = 1,2,3, is the disjoint union of finitely many connected compo- 
nents. Each component of is a smooth real submanifold of O of half the dimension. Each 
component is stable under the dilation action o/M^ on 3. 

is a symplectic real form o/(0,I, S) and P^ is a holomorphic Lagrangian submanifold 
o/(C,I,S). 

Let be the subgroup of U commuting with then U"^ is the subgroup with Lie algebra 
equal to a. 

Let TZ € 50(3) be the element which gives the cyclic permutation 

7^(ai + 6j + ck) = ci + aj + 6k (6.21) 
Then TZ defines an automorphism TZ : M{k) ^A{K) by ( |5.16| ). 
Corollary 6.8 [Vej We have (M^ x U^)-equivariant diffeomorphisms 

^b^-^ : P" P'' (6.22) 
This sets up a bisection between the connected components of P^ and those of P^ . 
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We get the following commutative diagram of diffeomorphisms where (abc) is a cyclic 
permutation of 1,2,3: 



M{k) 



M{k 



pa pb 

Proof We have 

n-iA,it),A2it),Asit)) = iAs{t),A^it),A2it)) 
and so we get the commutative diagram 



Now ( 6.23 ) follows by restriction. 



(6.23) 



(6.24) 



(6.25) 



□ 



7 Real Nilpotent Orbits and the Vergne DifFeomorphism 

From now on, we single out in Theorem ^]^(iii) the holomorphic symplectic isomorphism 

$1 : M{k) O (7.1) 

Now by means of <I>i we transfer the hyperkaehler structure on M{k) over to O. Thus we 
get a set 



(5, 1, J,K,u;i,a;j, wk) 



(7.2) 



of hyperkaehler data on O with a corresponding 2-sphere S of complex structures on O. We 
call this the instanton hyperkaehler structure on O. 

By Theorem I = (<I>i),,Ji is the natural complex structure on O discussed in §^ and 
the holomorphic symplectic KKS form is 



S = cjj + iiOK 



(7.3) 



The three moment maps Cij C2; C3 • A4(«;) u transfer by means of $1 to three [/-equivariant 
moment maps 



Ci : O ^ u, Cj-O^u, Ck-.O^u 
We recover the natural embedding ^ : O ^ q from two of these maps as 

^' = Cj + <K 
However the third moment map Cl is a mystery. 



(7.4) 



(7.5) 



Instantons and Kaehler Geometry of Nilpotent Orbits 



23 



The S'0(3)-action ( 5.16| ) on M{k) transfers by means of $i to a smooth action on O 

50(3)^DiffO (7.6) 

which commutes with the [/-action. We caU this the Kronheimer SO{3)-action on O. 

We emphasize that this 50(3)-action depends on a choice of maximal compact subgroup 
of U of G. The Kronheimer S'0(3)-action remains quite mysterious. 

We can rewrite (|6.6|) as 

gK = «KepR (7.7) 

where = a and pK = ib. This is a Cartan decomposition of the real semisimple Lie algebra 
0R in the usual sense: 6k is a maximal compact Lie subalgebra of and pK is an ad fiR-stable 
complementary subspace. Then (|6.2D becomes 

u = e ipR (7.8) 

In practice, one may start from (7.3.1) and then constructs u by (|7.8| ). Then and pR are 
real forms of t = o © io and p = b © ib and the complexification of ( |7.7| ) is 

= «©p (7.9) 

Let ( , )gj. be the Killing form of gu; this coincides with the restriction to qr of the complex 
Killing form ( , )g. 

We know that <I>i identifies M{Ky^ with O n by ( |6.12| ) or Proposition |6]^. Let Cr 



be a connected component of C Then Or is stable under the dilation action of on qr. 
Moreover Or is an orbit under the adjoint action of Gr. This follows easily since Or is a 
connected real form of O and O is an adjoint orbit of G. 

Hence Or is a real nilpotent orbit of Gr. Every real nilpotent orbit arises in this way. 
Let a be the real GR-invariant KKS symplectic form on Or. Let 

: Or ^ 0M (7.10) 

be the natural embedding. Then a is the unique symplectic form on Or such that the adjoint 
action of Gr on Or is Hamiltonian with moment map (j). In terms of the component function 
(j)'^, w € 0R, defined by 4>'^{z) = {'w,z)g^, this means that the fi-Hamiltonian flow of the 
functions cp"^ gives the GR-action and the map 

0r^G°°(Or), w^r (7.11) 

is a Lie algebra homomorphism with respect to the Poisson bracket on G°°(Or) defined by 
a. We easily see that 

Lemma 7.1 The real nilpotent orbit (Or, a) is a symplectic real form of the complex nilpotent 
orbit (0,I,S). I.e., Or is a real form of O and the real part ofTi restricts to a on Or while 
the imaginary part of S vanishes on Or . Thus 

a = (7.12) 
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Theorem 7.2 is a complex submanifold of O with respect to the complex structure J on 
O. Consequently the Kaehler structure (g, J,u;j) on O defines by restriction a K^-invariant 
Kaehler structure on Or. 

The Kaehler form coincides with the KKS symplectic form a. 

We call this Kaehler structure on Or the instanton Kaehler structure. 



Proof By (|7.12D and dTJ), we have 

a = ReS|og = Re(a;j + iuj-K)\o^ = wjIor 



The rest is an immediate consequence of our work in Indeed, by Corollary ^]^(iv), M.{ 
is a J2-holomorphic complex submanifold of M{k). We have <I>*J = J2 by the definition of 
J. So <^i{M{k)"''^) is a J-complex submanifold of O. But we saw <^i{M{Ky^) = O n 0ir in 
( |6.12 ). Thus each connected component of O fl is a J-complex submanifold of O. □ 



Remark 7.3 Here is another proof that O fl gig is a J-complex submanifold of O. We can 
transport the diffeomorphism TZ : Jv[[k) — s- Jv[[k) to a diffeomorphism 7^ : O — > O by means 
of $1. Then Proposition ^ and Corollary |^ imply that Or\Q^ = 7^(0 np). But Onp is an 
I-complex submanifold of C So O n 0ir is a complex submanifold with respect to TZiV) = J- 



Example 7.4 We illustrate Theorem [7.2| in the context of the example of flat hyperkaehler 



space discussed in Examples |2.1| , p.2| , and 4.3. The case n = 1 is sufficient to show how this 
works. 

Let O C s[(2, C) be the non-zero nilpotent orbit of SL(2, C). We have a covering map 
vr : - {0} ^ O, 7r(a + bi + cj + dk)= ( "2 



uv — 
v'^ —uv 

where u = a + bi and v = c+di. Let Cm be the Euclidean connected component of C'ns[(2,M) 
containing 7r(l + j). For q = a + b\ + c} + d\^ we find that 

7r(g) G Or <S=^> M^jMf,^^ G M and + > u,?;GM 6 = d = 

Thus 

7r-i(OM) = (R + Mi)-{0} 

Plainly, (M -|- Mj) — {0} is a complex submanifold of EI — {0} with respect to itJ (and only 
±J). But TT is a covering of hyperkaehler manifolds, and so it follows that Or is a complex 
submanifold of EI — {0} with respect to it J (and only it J). 

We have a source of J-holomorphic functions on O, namely the component functions $j, 
z € 0, of the J-holomorphic moment map <I>j : O — > g. We next examine how these restrict 
to Or. For this, we recall the complex Cartan decomposition (|7.9|). 



Theorem 7.5 The functions x € vanish identically on Or. However for f € p, the 
functions 

r = mw (7.13) 
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separate the points of Or and the corresponding holomorphic map V : Or p defined by the 
f" , so that {v^V{w))q = f'"{w), is a locally closed embedding. 

The image Y = V{Om.) is a single K -orbit in p. Y is a connected component ofOCip and so 
in particular Y is stable under the dilation action ofC*. The resulting (K^ xM'^)-equivariant 
dijfeomorphism 

V:Or^Y (7.14) 

is the Vergne dijfeomorphism. 



Proof We know by Proposition |6.6| that $j gives a diffeomorphism of O H onto O Hp. 
This means that ^j, x € t, vanishes on O n 0ir while the functions ^j, v £ p, embed O H 
into p as O dp. But then the functions i^j do the same thing. The diffeomorphism 

V:OngK^Onp (7.15) 



defined by i^j is the Vergne difi'eomorphism discovered in |Ve]. 

Now Y = V(Ck) is a connected component of Of) p. We know by a well-known argument 
that Y is i^- homogeneous. Indeed, K acts on O n p and at any point e G 1", we have a 
complex Lagrangian decomposition 

TeO=[Q,e] = %e]®[p,e] (7.16) 

Also [i,e] C p while [p,e] C t. It follows that TeY = [i,e]. So the JC-action on Y is 
infinitesimally transitive and hence transitive. □ 



Remark 7.6 (i) We introduced the factor i in ( [i".13| ) for convenience (later in Theorems 
|7.9| and |9.3| ) Essentially, the factor i arises because it is iO^, not Ok, which most naturally 
corresponds to Y in the Sekiguchi correspondence. 

(ii) The Vergne difi'eomorphism in ( |7.15| ) recovers and explains the Kostant-Sekiguchi 
correspondence [^ . 



Corollary 7.7 The K^- action on Ok complexifies with respect to 3 to give a transitive holo- 
morphic action of K on Ok- Then the Vergne dijfeomorphism ( 7.14 ) is K-equivariant. 

Theorem |7.5| and Proposition |6.6| give 



Corollary 7.8 Let AgM{k). Then 

$i(A)gOk ^ MA)£Y 

Let 



$3(^) G iOw 



(7.17) 



(7.18) 



be the projection map defined by the Cartan decomposition ([7?7|). Then /x is a i^K-equivariant 
moment map for the /CK-action (inside the Gig-action) on O. 
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Theorem 7.9 Triple Sum Formula. Let w € Ok. Then the Vergne dijjeomorphism ( |7.14| ) 
satisfies 



w = li{w) + -V{w) + -V{w) (7.19) 

In ( [7.19D , we are taking the sum inside g of the three vectors. In view of ( [7.7D , (|7.19D says 
that \V{w) + \V{w) is the projection of w to p]R. 

Proof Given w € Ok, let A € be the (unique) instanton such that 

w = ^M) = + iUA) (7.20) 



By Proposition |6.6| , we have (taking a = ^k, b = 

Ci{A) G ipR, C2(^) G C3(A) G ipR (7.21) 
So the projection of w to £m is 

/i(tx;) = C2{A) (7.22) 

Now 

V(ii;) = i^2{A) = i{UA) + KM)) = -CM) + iCM) (7.23) 
Since C,i{A) G ipR is pure imaginary and iC,z{A) € pM is real, we get 



V{w) = CM) + KM) (7.24) 

So 

]^(y{w) + V{w))=iCM) (7.25) 



Now ( 7.19 ) is immediate. □ 

The real symmetric pair (g^, 6]g) (or the complex symmetric pair (g, ^)) is called Hermitian 
if the corresponding real symmetric space G^/K^ has a i^R-invariant Hermitian structure. 
This amounts to the condition that we can find xq G Cent 6r such that the eigenvalues of 
adxo on p are ±i. Then we get a -fCiR-stable splitting 

p = p+ ep" 

where p^ is the iti-eigenspace. Then dime p^ = dime p^ and moreover p+ and p~ are 
mutually contragredient K]R-representations. 

Lemma 7.10 Suppose (fl,^) is Hermitian. Then the following two M-linear maps are in- 
verses: 

PM-^P^, u ^{u - i[xo,u]) and p+ ^ pK, v v + v 
The Addition Formula recovers in the Hermitian case the fact 



Corollary 7.11 |Ve| , Prop. 6] Suppose (0,6) is Hermitian and Ok is such that Y = V(Ok) 
lies in p+. Then the Vergne diffeomorphism V : Ok Y is given by the composition 

Or "-^ 0R — ^ pR P"^ 
where the middle map is the projection defined by the Cartan decomposition ( [7.71) . 
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8 The KV Circle Action on O^. and the Kaehler Potential po 

The instanton Kaehler structure (J, a) We let d = 9 + 5 be the canonical splitting of d into 
its J-holomorphic and J-antiholomorphic parts. (So d = dj and d = dy) 
Corollary |5.4| and Corollary ^]^(ii) give 



Corollary 8.1 The complex nilpotent orbit O, equipped with its U -invariant instanton hy- 
perkaehler structure, admits a U -invariant hyperkaehler potential p : O — > M, unique up to 
addition of a constant. The further condition that p is homogeneous of degree 1, with respect 
to the Euler scaling action o/M^" on O, determines p uniquely. 

Let 

Po-.Ok^R (8.1) 
be the smooth function obtained by restricting the hyperkaehler potential p from Corollary 



8.1. Plainly Theorem pHl and Corollary pH^ give 



Corollary 8.2 po is a global Kaehler potential on Or so that 

a = iddpo (8.2) 

The function po is K^-invariant and homogeneous of degree 1 with respect to the Euler scaling 
action o/M^ on Or. 

Next we consider how the Kronheimer S'0(3)-action on O from ^ transforms the sub- 
manifold O^. 

Proposition 8.3 The circle subgroup Cj of 50(3) preserves Or and so defines a smooth 
group action 

U:S^^ Diff Or (8.3) 

We call this the KV (Kronheimer- Vergne) -action, and we let 6 denote the infinitesimal 
generator. 

The KV -action on Or is free, commutes with the K^-action and is Kaehler. Moreover 
the KV S^- action is Hamiltonian and po is an equivariant moment map. I.e., 

e_ia + dpo = (8.4) 

or, equivalently. The Hamiltonian flow of po is the KV -action. 

We write the KV S'^-action as, for w G Or, 

n(e^*)(u;) = e^**u; (8.5) 

Proof The first statement is clear by, e.g.. Corollary 6.4. The properties of the KV 5^-action 
are then immediate from the corresponding properties of the Kronheimer 50(3)-action on O 
or M{k). □ 

The J-holomorphic functions ^j, z (z Q, on O transform under the action of Cj by the 
degree 1 character. Consequently Proposition gives 
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Corollary 8.4 For v € p, the J -holomorphic functions f" on Ok satisfy 

{Po,n = ir (8.6) 

We get insight into the KV S"^ -action by trying to find it inside the action of K^. For 
simplicity of exposition, we assume in the next result that g is simple. If g is simple, then 
either 

(I) (g,^) is Hermitian and CentfiiR = Mxq where xq 7^ was chosen in §7.9, or 

(II) (g,t) is non-Hermitian, Cent 6k = and p is irreducible as a ^CR-representation. 

Proposition 8.5 Suppose q is a simple complex Lie algebra. Then 

(I) // (g, i) is Hermitian and Y C then the KV -action on Ok is given by the center of 
i^K cind 

Po = <^^"° (8.7) 

(II) Otherwise, the KV S^- action on Ok ^^es outside the action o/Gk- 

Proof To prove that po = where c is constant, it is necessary and sufficient to show 
that 

{Po,n = {<^'="",n (8.8) 

for all V € p. This is because the functions f", v & p, separate the points of Ok and both po 
and (p^° are M"'"-homogeneous of degree 1. 

Let V ep. Then = f^'"-''''^ and so in the Hermitian case we get 



We want to compare this with (8^). If y C p"*", then f^ vanishes on Y for all u € p^. For 
V E p^, ( ^.^ ) holds with c = — 1. So then po = (j)'^^ . Similarly Y <Z p^ gives po = f/*^". In 
either case, it follows that Cent-fCK gives the Hamiltonian flow of po, and this is just the KV 
circle action. 

Now, continuing the Hermitian case, suppose Y fails to lie in p+ or p^. Then we cannot 
find c € M such that (|8.8| ) holds for all f € p. Since po is i^'K-invariant and (up to scaling) (p^'^ 
is the only i^K-invariant function in (/)*(gK), we see that po lies outside of (/>*(gK)- If (fl)^) is 
non-Hermitian, then has no non-zero i^'K-invariants, and so certainly />„ lies outside 

of (t)*{m)- 

The condition that po lies outside of (t>*{%m.) means exactly that the Hamiltonian flow of 
Po lies outside the action of Gk. n 



Using Po, we next prove in Theorem ?.(: below that the Vergne difFeomorphism produces 
an embedding of Ok into T*Y which realizes T*Y as a symplectic complexification of Or. 
Let 

s:Ok ^ Yxl^ Yxt (8.10) 

be the smooth map built out of the Vergne difFeomorphism, the moment map p and the 
obvious inclusion ^k ^ 6- Then s is an embedding of Ok into Y xi, since already V is 1-to-l. 
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On the other hand, we have a natural holomorphic embedding 

T*Y^Yxt (8.11) 

of the cotangent bundle T*Y of Y. Indeed, differentiating the iiT-action on Y we get an 
infinitesimal holomorphic vector field action 

t^23ect^°' Or, x^r?^ (8.12) 

The induced ivT-action on T*Y is holomorphic Hamiltonian with momentum functions given 
by the holomorphic symbols of the vector fields rj^. Let M : T*Y ^ 6 be the corresponding 
holomorphic moment map. Then ( 8.11 ) is the product of the canonical projection T*Y Y 
with M. It follows since the ii'-action on is transitive that the product map is 1-to-l. 

Theorem 8.6 The image of s in Y x t lies inside T*Y. Furthermore the resulting K^- 
equivariant map 

s:Ou^ T*Y (8.13) 
embeds (Ok, a) as a totally real symplectic submanifold of {T*Y,0,) so that 

s*{Ren) = a and s*(ImJl) = (8.14) 
where is the canonical holomorphic symplectic form on T*Y . 

Proof We may regard the vector fields ry^ on Y as vector fields on Or using the Vergne 
diffeomorphism. Showing that the image of s lies in T*Y amounts to showing that // : Or — > 
t]R is given by a real smooth 1-form (3 on Or in the sense that for all a; € we have 

r/^) = /i^ and {P,Jvl = (8.15) 



This follows easily because, since K acts transitively on Or by Corollary 7.7, the holomorphic 
tangent spaces of (Or, J) are spanned by the vector fields r/^, Jrj^, x G Jr. 
Furthermore, we find a 1-form f3 giving /i, then 

s*(Ree)=/3 (8.16) 

where is the holomorphic canonical 1-form on T*Y. Then dQ = Q and so (8.7.5) holds if 
and only if d(3 = a, i.e., if and only if /3 is a symplectic potential. 

So the problem is to produce a symplectic potential (3 on Or satisfying ( 8.15| ). We claim 
that 

P = -'^{d-d)po (8.17) 

works. First /3 is a symplectic potential since 

dp = -^{d + d){d-d)po = iddpo = a (8.18) 

It follows that the functions (/?, ??^) are momentum functions for the XR-action on Or. Con- 
sequently, for each x S tm., {(3,r]^) is equal to p'^ up to addition of a constant. But both 
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functions {(5,r}^) and /i^' are homogeneous of degree 1 with respect to the Euler R^-action, 
and so {P,r]^) = fi^. Finally, we find 

{(3,Jvn = (J/3,7?-) = ^{(d + d)po,vn = l{dpo,vl = Iv'iPo) = (8.19) 



since Jd = id, Jd = —id and po is KK-invariant. This proves (8.14) and it follows easily that 



Offi is a totally real submanifold of T*y. □ 
Corollary 8.7 The embedding s realizes {T*Y, 0,) as a symplectic complexification o/(Oir, a). 

9 Toward Geometric Quantization of Real Nilpotent Orbits 



We can interpret Theorem 7.2 as providing a polarization on Or in the sense of Geometric 
Quantization. 

Corollary 9.1 Ok, equipped with its KKS symplectic form a, admits two transverse K^- 
invariant complex polarizations, namely the holomorphic and anti-holomorphic tangent spaces 
defined by the instanton Kaehler structure. 

Remark 9.2 We have two rather different extensions of the Kaehler X]R-action on Ok to 
a transitive action of a larger group. These extensions are given by the G]R-action and the 
action. But neither of these larger actions is Kaehler. Indeed the G]R-action is symplectic 
but not holomorphic, while the X-action is holomorphic but not symplectic. 

The instanton Kaehler structure on Or provides the first step in our quantization program 
for Or. In the spirit of Geometric Quantization (GQ), the quantization problem on Or is 
to convert R-valued smooth functions (j) on Or into Hermitian operators Q{(j)) on a Hilbert 
space Ti. of J-holomorphic half- forms on Or. This conversion must satisfy the Dirac axiom 
that the Poisson bracket of functions goes over into the commutator of operators so that 

Q({0,V'}) = i[Q(0),Q(V')] (9.1) 

By the well-known No-Go Theorem, this conversion cannot be carried out for all smooth 
functions (or even for all polynomial functions on a real symplectic vector space) consistently 
so that (|9.1| ) is satisfied. However, we expect (for various reasons) that the Hamiltonian func- 



tions w G in ( 7.11 ) can be quantized consistently, modulo some "isolated anomalous" 
cases. 

Quantization of the functions t« S 0m, already would solve the Orbit Method problem 
in representation theory of attaching to Or an irreducible unitary representation (or finitely 
many such representations) of the universal cover of Gr. Indeed, the operators iQ{4y^) define 
a Lie algebra representation of qr on the space of KR-^im.te vectors in Ti. 

The larger GQ problem is to construct the operators Q{(j)^) and the Hermitian positive 
definite inner product on (some subspace) of holomorphic half-forms that gives rise to 7i. 

The XK-invariance of the instanton Kaehler structure means that the Hamiltonian flow 
of the functions (p^ , x ^ Ir preserves J. Thus the Lie derivative of the Hamiltonian vector 
field ^(f,x = rj^ gives us a natural choice for the quantization, namely 

Q((/>-) = xetR (9.2) 
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But the Hamiltonian flow of the "remaining" functions (p'" , v G pK, does not preserve J. 
Our strategy for quantizing these remaining functions is to "decompose" them in terms of 
holomorphic and anti-holomorphic functions on Or. The reason for this is that we expect 
that a holomorphic function / quantizes to multiphcation by g, while an anti-holomorphic 
function g quantizes to the adjoint of multiphcation by /. (Another Dirac axiom is that Q{f) 
and Q(/) are adjoint.) Here Q{f) = f and = Q{g)* are densely defined operators on 

n. 

A decomposition of (j)^ with respect to holomorphic and anti-holomorphic functions should 
be of the form (p^ = Yl fp9p where fp and gp are holomorphic functions. Of course, the sum 
could well be infinite and unwieldy to quantize. 

The key point is that the decomposition of (p^ is remarkably simple: 

Theorem 9.3 Let v G p]R. Then the Hamiltonian function (jf : Ok ^ M zs the real part of a 
J -holomorphic function on 0]r. In fact 

<A'' = Rer = ^(r + r) (9.3) 



where f" is the J -holomorphic function defined in Theorem 7.5. Moreover 

f- = r-i{po,n (9.4) 



Proof The first part is a corollary of the Triple Sum Formula Theorem 7.9. Indeed, let 
w € Or. Then using ( 7. 191) we find 



2cP^iw) = 2iv, w), = {v, V{w) + Vjw)), = jv, V{w)), + {v, V{w)), 

= nw)+f-{w) 

Next, we have {—ipo, f^} = f " by ( |8.6| ) and so {—ipo-, f""} = —f^ by complex conjugation. 
Then (|9^ gives 

{-ip^,c^^} = ^(f--p) (9.5) 

Now adding dJ) and dJ) we get □ 

We may choose a set zi, . . . ,Zn of local holomorphic coordinates on Ok. Then (j)"" is a real 
analytic function in the 2n coordinates zi, . . . , Zn,zi, . . . ,Zn and (|9.3|) says that 



(p^izi, . . . , Zn, Zi,..., Zn) = fv{zi, . . . , Zn) + fv{zi, ■ ■ ■ , Zn) 



Corollary 9.4 The functions (j)" , v G p^, on Ok are pluriharmonic. 



Remark 9.5 (i) Alternatively, we can prove ( |9.3D right after Theorem [7.2| in the following 
way. We find for u € pK 
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because ( p. 16 ) gives <I>| = —i^^ = —iCj' + Ck • But also we have the J-holomorphic function 
So (p"" = ReT where f = i^"^. 

(ii) The condition (p^ = Re determines a holomorphic function uniquely up to addition 
of a constant. The additional condition that F'" is homogeneous then forces F^ = ■ 

We summarize the way the KV 5^-action produces the complex structure J on in the 
next Corollary. Let 

D = (/'*(pK) = {c/>n^;GM (9.6) 
so that D is the space of Hamiltonian functions on corresponding to pK. 

Corollary 9.6 Let C C°^(C']r) he the subspace spanned by all the translates of d under 
the KV -action. If Q is simple then in Cases (I) and (II) of Proposition we find 

(I) = V and so D ~ pK as K^-representations. 

(II) We have the direct sum 

O^ = oe{po,o} (9.7) 

and so D ~ pM © pM as K^-representations. 

In either case, decomposes under the KV action of ~ 5*0(2) into a direct sum of 
copies of the 2-dimensional rotation representation so that ~ © • • • © M^. Consequently 
the complexification dc splits into the direct sum 

dc = Oc ® "^c (9-8) 

where d^ = {(/> G d | e*^ * = e^^^cj)}. In Case I, d^ identifies with p^. 

In either case, dj and are complex- conjugate K^-stable spaces of complex-valued Pois- 
son commuting functions on O^. We have 

d+ = {ri?;Gp} and dc={7^|t;Gp} (9.9) 

Finally J is the unique complex structure on such that the functions f", v £ p, are 
J-holomorphic. 



Remark 9.7 Corollary |9.6| says in particular that 

d" = d + {po, d} + {po, {po, d}} + • • • 
Moreover, we could take this as the definition of d", and then all the assertions in Corollary 



9.G read the same. 
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A Appendix: Real Algebraic Varieties and Nash Manifolds 



A.l Introduction 

In this appendix we present some basic notions from the theory of real algebraic varieties. 



Some references are [ BoCR |, [ BeR |, [ BoE 



On many points, we follow the treatment in [BoCR|. We add material about real algebraic 
variety structures associated to complex algebraic varieties. Our discussion of Nash manifolds 
and Nash functions is more general. The last subsection § A.10 explains the application of 
this theory to real groups and their orbits. 

We present this material here for lack of an appropriate reference in the literature. 



A. 2 Real Algebraic Sets 

A subset V C M" is a (real) algebraic set if V is the set of common zeroes of some finite set 
of real polynomial functions P G M[xi, . . . , Then a function / : y — >■ M is called regular 
if there exist polynomial functions P,Q& M[xi, . . . , x„] such that Q has no zeroes on V and 
f{x) = P{x)/Q{x) for all x . The set of regular functions on V forms an M-algebra which 
we win caU A{V). 

Let P{y) C A{y) be the image of the natural ring homomorphism M[3;i, . . . , Xn] A.{V). 
Then 



P{V)c^R[xi,...,Xn]/I{V) 

where I{V) C PiV) is the ideal of polynomial functions vanishing on V. Notice that A{V) 
is algebraic over P{V); indeed, if / = P/Q then Qf — P = 0. 

A map (j) : V ^ V', where V' C is an algebraic set, is regular if each of the component 
functions (pi, . . . ,(prn is regular. An equivalent condition is that the pullback of a regular 
function on V' is regular on V; then (p induces an algebra homomorphism (p* : A{V') A{V). 
Conversely, any algebra homomorphism p : A{V') — > A{V) defines uniquely a regular map 

V v. 

An isomorphism of algebraic sets is a bijective biregular map. Isomorphisms V ^ V' are 
in natural bijection with algebra isomorphisms AiV') A{V). 

We may also use the term real algebraic in speaking of regular functions and maps. 



A. 3 Real AfRne Algebraic Varieties 

The Zariski topology on an algebraic set V C is defined just as in the complex case, so 
that the Zariski closed sets in V are precisely the algebraic sets in which lie in V. This 
topology is not Hausdorff but it is Noetherian and hence quasi-compact (every open cover 
has a finite subcover) as the polynomial ring R[xi, . . . , Xn] is Noetherian. 

Every Zariski closed set is closed in the usual Euclidean topology on defined by the 
Euclidean metric, as polynomials are continuous. We will refer to open sets, closed sets, etc 
as "Zariski" or "Euclidean" to distinguish the two topologies. 

A topological space M is called irreducible if M cannot be written as the union of two 
closed subsets different from M. We say an algebraic set is irreducible if it is irreducible in 
the Zariski topology. 
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A regular function on a Zariski open set U G V is one of the form P{x)/Q{x) where 
P,Q & M[a;i, . . . , Xn] and Q is nowhere vanishing on U. The set of regular functions on U is 
closed under composition and forms an R-algebra which we will denote Av{U). 

The assignment U hh- Av{U) defines a sheaf Ay of M- algebras on V with respect to its 
Zariski topology. In particular, if Ui, . . . , Um is a finite Zariski open cover of a Zariski open 
set U C V and / is a real- valued function on U such that f\u. = Pi/Q-i then we can find 
P,Q ^ M[xi, . . . , Xn] such that Q is nowhere vanishing on U and f = P/Q. Indeed, as Ui CV 
is open, the complement V — Ui is the zero-locus of a finite set of polynomials; let Fi be the 
sum of their squares. Then Ui = V H (F^ 0) and the polynomials P = YT ^iQi^i and 
Q = Q^Pf satisfy our requirement. 

Then (V, Ay) is a ringed space in the usual sheaf theory sense. 

Now we can define an (abstract) real affine algebraic variety: this is a pair {X,Ax) where 
X is an irreducible topological space, Ax is a sheaf of R-algebras of R-valued functions 
on X and there exists an isomorphism of ringed spaces from {X,Ax) to iy,Av) for some 
(irreducible) real algebraic set V. 

If 5 C X is closed then S identifies with an algebraic set of inside V. In this way, 
if S is irreducible, S acquires a canonical real algebraic affine variety structure; we call the 
corresponding structure sheaf Ax,s- 

A. 4 Real Algebraic Varieties 

In complete analogy with the complex case, real algebraic varieties are obtained by gluing 
together affine ones. 

A real algebraic variety is a a pair (X, Ax) where X is a Noetherian irreducible topological 
space and Ax is a sheaf of R-valucd functions on X satisfying this condition: there exists 
a finite open cover {Ui}i^i of X such that for each i, the ringed space {Ui,Ax\ui) is a real 
affine algebraic variety. Then Ax is called the structure sheaf of X. The sections of Ax are 
the regular functions on U. The topology of X is then called the Zariski topology. 

In speaking of real algebraic varieties, we may omit the modifiers "real" or "algebraic" 
when the context is clear. (However, often we will be dealing with complex algebraic varieties 
or real analytic manifolds at the same time.) 

A regular mapping between varieties (X, Ax) and (y. Ay) is a Zariski continuous mapping 
(j) : X such that ii U d Y is open and / G Ay{U) then </.*/ = / o G Ax{(l}~^U). 

An isomorphism is a bijcctivc biregular map. We often speak of X as the variety and leave 
implicit its structure sheaf Ax ■ 

An affine open set U of X is then a Zariski open set U such that {U,Ax\u) is an affine 
variety. 

Let X be a a real algebraic variety Then we have the following examples of real algebraic 
subvarieties of X. 

(i) Suppose W is Zariski open in X. Then W is again a variety where we define Aw by 
restriction of the structure sheaf of X. 

iii) Suppose S is an irreducible Zariski closed set in X. Then S is again a variety where 
for each open affine set U (Z X we have As{S fl C/) = Ax,s{S fl U). If X is affine then so is 
S. 

(iii) Now (i) and (ii) imply that any Zariski locally closed irreducible subset of X is 
again a variety. For such a subvariety we may write .4.x liy for the induced structure sheaf 
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Aw- A regular map (p : X ^ Y oi varieties is a locally cAosed embedding if (j){X) is a locally 
closed subvariety of Y and (p defines an isomorphism {X,Ax) ^ •^Y\(f>{x)) ■ 

A. 5 Real Structures and Real Forms 

A complex algebraic variety Z is defined over M if Z is equipped with an involution 

K : Z ^ Z 

called complex conjugation, which satisfies the following: Z admits a cover by complex 
algebraic At-stable afHne open subsets U such that 

(i) if / G R{U) then the function / defined by f{u) = f{K{u)), u e U, also lies in R{U), 

and 

(ii) the map R{U) —^ R{U), / h^- /, is an M-algebra involution of R{U). In other words, 
the real subspace {/ G R{U) | / = /} is both a real form and a real subalgebra of R{U). 

We call this collection of real forms, or k itself, a real structure on Z. We write z = k{z) 
for z ^ Z. Geometric objects on Z such as functions, vector fields and differential forms are 
defined over M, or real, if they are stable under complex conjugation. 

If / : Z — > is a complex algebraic morphism of complex varieties defined over R, then 
/ is defined over M, or real, if / commutes with complex conjugation. 

Let Z'^ be the set of real, i.e., At-fixed points in Z. We put Z{R) = Z'^. Suppose Z(U) 
is non-empty and Zariski irreducible. Then Z(M) has a natural structure of real algebraic 
variety and we call Z{R) a real form of Z. 

To see this, we first define a Zariski topology on Z(M) by the collection of sets 

S = {[/(M) = [/ n Z(M) \U eT} 

where T is the collection of K-stable Zariski open subsets of Z. For each affine W ^ S we 
define Az(K) {^) to be the space of quotients P/Q where P and Q are regular functions on 
some U & T with U{M) = W, P and Q real (i.e., K-fixed) and Q nowhere vanishing on W. 
This data defines a unique sheaf Az(m.) of M-valued functions on Z(R) and then {Z{R),Az(m.)) 
is a real algebraic variety. 

Notice that Z{M.) is affine if Z is affine. In fact suppose Z C C" is defined by the vanishing 
of Pi, • • • , Pm € C[zi, . . . ,Zn] and also Z is complex conjugation stable. Then Z(R) is the 
zero-locus in of the 2m real polynomial functions defined by the real and imaginary parts 
Re(Pi), Im(Pi), . . . , Re(P„), Im(P„) e R[2i, . . . , z,,]. 

Clearly every real affine algebraic variety is of the form Z(R) for some complex affine 
algebraic variety Z defined over R. 

The process Z f— > Z(R) is compatible with the usual operations on varieties. For instance, 
if : Z ^ Z' is a regular map of complex algebraic varieties defined over R, then the induced 
map (/>(R) : Z(R) Z(R)' is a regular map of regular algebraic varieties. 

Real structures often arise in the following way. Suppose F is a complex vector space 
and is a real form of V with corresponding complex conjugation map k :V ^V. Then k 
defines a real structure on every K-stable (locally closed) complex algebraic subvariety X of 
V. 

Suppose a complex algebraic group H acts on Z and H is defined over R. We say the 
H-action on Z is defined over R if the action morphism H x Z ^ Z is defined over R. This 
happens if and only if for every h e H, the transformations of Z defined by h and h are 
complex conjugate. 



36 



R. Brylinski 



A. 6 The Complex Conjugate of a Complex Variety 

Given a complex algebraic variety Z, we may construct another complex algebraic variety 
Z called the (abstract) complex conjugate variety. If Z is affine, then Z is the unique affine 
variety such that 

R(Z) = RiZ) 

where R{Z) is the C-algebra which is complex conjugate to R{Z); i.e., R{Z) has the same 
underlying R-algebra structure but has the complex conjugate complex vector space structure. 
For general varieties, Z is defined in the obvious way by gluing together complex conjugate 
affine opens. 

If / : Z — > is a morphism of complex varieties then the complex conjugate map 
J:Z^W defined by 

7{p) = m 

is also a morphism. 

The construction of Z from Z is functorial in the usual ways and commutes with products. 
We have natural identifications TZ = TZ and T*Z = T*Z for the holomorphic tangent 
and cotangent bundles. Also puUback of differential forms and pushforward of vector fields 
commutes with taking the complex conjugate. 

Consider the natural map 

Z ^ Z X Z, zi-^ {z,z) 

This embeds Z as a real form of Z x Z with respect to the real structure defined by {u, v) = 
{v,u). Thus in particular, Z itself has a canonical structure of real algebraic variety. This 
amounts to "forgetting" part of the complex algebraic variety structure. Notice that Z and 
Z acquire isomorphic real algebraic variety structures in this way. 

We may write Z^''^"''' for Z regarded as real variety. If Z is an affine complex variety 
then Z^^"' is just the obvious affine real variety. Indeed suppose Z C C" is defined by 
the vanishing of Pi, • ■ ■ , Pm G C[zi, . . . , Zn]- We have a natural R-algebra homomorphism 
C[zi, . . . , Zn] — > C[xi, yi, . . . , Xn, Vn], Say P I— > P', defined by setting Zj = Xj + iyj. Then 
^reai ^ ^2n -g ^]^g closcd real algebraic subvariety defined by the vanishing of the real and 
imaginary parts ReP{,ImP{, . . . ,Re P^,lm P^^ G R[xi,yi, . . . ,Xn,yn]- 

If Z has a real structure k, then the map 

Z —> Z, z ^ n{z) 
is an isomorphism of complex algebraic varieties . 

A. 7 Tangent Spaces, Dimension, and Smoothness 

Let V be a point of an irreducible algebraic set V C M". The Zariski tangent space T^V at 
V may be defined as the linear subspacc of W"" given by 

r„y = {x G I (gradP|„) • a; = for ah P G I{V)} 
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The dimension dy = dimTyV is generically the same over V (i.e., is the same over some 
Zariski open dense set of V). This common value of is caUed the dimension dy of V . A 
point f G y is a smooth point if dy = dy- The set V^^^ of smooth points is Zariski open 
dense in V . y is a smooth variety if y = V^^^ . 

These notions pass immediately to affine real algebraic varieties and then are purely local. 
These notions then pass to general real algebraic varieties as the latter are obtained by gluing 
of affine opens. In particular then the notions of Zariski tangent space and smooth point are 
purely local. In the usual way one defines etale maps of real algebraic varieties. 



If Z is smooth, then, in the context of § A^, Z(M) is a smooth real form of Z (in particular 



Z(M) is irreducible). This follows by observing that at each point z £ Z'(]R) the complex 
Zariski tangent space T^Z acquires a real structure and then the real points form the tangent 
space to the real submanifold Z(M). 

A smooth real algebraic variety X has a natural structure of real analytic manifold, just 
as a smooth complex algebraic variety has a natural structure of complex analytic manifold. 
In particular X has a larger topology, often called the strong or Euclidean topology^ which 
refines the Zariski topology. On M", this is just the usual Euclidean topology. 

Now X, while connected in the Zariski topology (since it is irreducible), may well fail to 
be connected in the Euclidean topology. This typically happens when taking real forms. For 
example, the familiar real form of C* is R*. Fortunately, the individual Euclidean connected 
components have a natural structure, namely each is a semi-algebraic real analytic subman- 
ifold. In fact, each component is a Nash manifold. We develop this notion in the rest of this 
Appendix. The starting point is semi-algebraic sets. 

A. 8 Real Semi-Algebraic Sets and Maps 

A subset S C M" is a real semi-alqehraic set if 5 is a finite union of sets of the form: 



{x G I Pi{x) = . . . = P„ = and Qi{x), . . . ,Qrn{x) > 0} 

where Pi, Qj £ M[xi, . . . , 

Suppose S C M"" and T C M™' are semi-algebraic sets. A map : 5 — > T is a semi- algebraic 
map if the graph of i;A is a semi-algebraic set in M""'"™'. A semi-algebraic map f : S ^ M 
is called a semi-algebraic function. It follows that (p is semi-algebraic if and only if all the 
component functions (pi, ... , (pm are semi-algebraic. 

Notice that a regular map of real algebraic sets in Euclidean space is in particular a 
semi-algebraic map of semi-algebraic sets. 

Semi-algebraic sets arise inevitably in the study of real algebraic sets. Indeed the image 
of an algebraic set under a regular mapping, even a linear projection of Euclidean space, is 
in general only semi-algebraic. Also the connected components (in the Euclidean topology) 
of an algebraic set are generally only semi-algebraic. 

The Tarski-Seidenberg Theorem says that under a semi-algebraic map, the image of a 
semi-algebraic set is semi-algebraic. Another result says that a semi-algebraic set has finitely 
many connected components (in the Euclidean topology) and each such component is semi- 



algebraic (see IpoCRI , Th. 2.4.5, pg 31]). 

Next we define semi-algebraic sets in varieties. Let {X,Ax) be a real algebraic variety 
and let S C X. If X is affine, then we call S semi-algebraic if for one (and hence every) 
closed embedding cp : X ^ M" of real algebraic varieties, the set (p{S) is semi-algebraic in M". 
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Now for general X we call S semi- algebraic if for every affine open U C X (or equivalently, 
for every member U of some affine open cover of X), the set S OU is semi-algebraic in U. 

Notice that if C X is a locally closed subvariety, then S OW is semi-algebraic in W. 

Now, generalizing the definition above, if C X and T CY are semi-algebraic sets in real 
algebraic varieties, then a map (/> : — > T is semi- algebraic if the graph of (p is semi-algebraic 
in X X y. It is routine to check that the Tarski-Seidenberg theorem is still true in this setting. 

A Euclidean open semi-algebraic set, and so in particular a Euclidean connected compo- 
nent, of a smooth real algebraic variety is a real analytic submanifold. 

An easy, but important observation is the following: if X is a real algebraic variety and 
(j) G ^(X) is such that (p takes both positive and negative values on X then the set 

5 = ((A > 0) C X 

is semi-algebraic in X (but not algebraic). 



A. 9 Nash Functions and Nash Manifolds 

Suppose that 5 is a (Euclidean) open semi-algebraic set in a smooth irreducible algebraic set 
V C M". 

A real analytic function / : S" ^ M is called a Nash function if / satisfies the following 
two equivalent conditions: 

(i) / is algebraic over the algebra P(y) of polynomial functions and 

(ii) / is semi-algebraic. 

The Nash functions form a Noetherian M-algebra Ny{S) algebraic over P{V), and fur- 
thermore Nv{S) is integrally closed if S is Euclidean connected — see [ BoE|1 . 



From now on assume, more generally, that S is a semi-algebraic real analytic smooth 
submanifold of a smooth real algebraic variety X. 

If X is affine, then the definition above of Nash function on S and the equivalence of the 
two conditions go over immediately as soon as we replace (i) by the condition: 

(i') / is algebraic over A{X). 
(In the case X = V, this is consistent with the previous definition as ^(T^) is algebraic over 
P(V)) The Nash functions on S form a Noetherian M-algebra Nx{S) which is algebraic over 
^(X) and, if S is Euclidean connected, integrally closed. 

Now we can treat the case where X is not necessarily affine. A real analytic function 
/ : 5 — > M is a Nash function if / satisfies the following two equivalent conditions: 

(i) for each affine open U C X (or equivalently, for every member U of some affine open 
cover of X), the restriction f\snu is algebraic over A{U) and 

(ii) / is semi- algebraic. 

It follows from the affine case that the Nash functions form an A(X)-algebra Nx{S) which 
is integrally closed if S is Euclidean connected. 

Next we define the sheaf Ms of Nash functions on S. We start with the Euclidean topology 
on S. The collection J^s of semi-algebraic Euclidean open sets in S is a basis of this topology 
(e.g, use small open balls). If C/ € Ts, then we define Ms{U) = Nx{U). This data determines 
uniquely the sheaf Ms of R-algebras on S. 

The pair (S,Ms) is then an example of a Nash manifold. We will not develop a more 
general theory of Nash manifolds here as these examples are sufficient for purposes of studying 
orbits of real algebraic groups, as explained in § A.1[1| below. 
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In particular, smooth real algebraic varieties are Nash manifolds and all real algebraic 
constructions on them or among them are Nash in the sense discussed below. 

Notice that our constructions on S have nice functorial properties. For example, if X C X' 
is a (locally closed) embedding of smooth real algebraic varieties, then X and X' define the 
same Nash manifold structure on S. 

Now suppose S' C S is such that S' is a semi-algebraic real analytic smooth submanifold 
of X. Then S' with its sheaf A/'s'' of Nash functions, is a Nash submanifold of S. In particular, 
each Euclidean connected component of S is an open Nash submanifold. 

If (5", A/5) and {T,Mt) are two Nash manifolds, then a morphism of the ringed spaces is 
called a Nash map or a Nash morphism. Thus a map (/) : 5 — > T is Nash if and only if for 
each Euclidean open set V C T, (l)-^{V) is Euclidean open in S and (p defines an algebra 
homomorphism (/)* : AfxiV) N's{4'^^{V)) by pullback of functions. A Nash map is a 
Nash isomorphism if (j) is bijective and (/)~^ is Nash. 

A Nash map (/> : S" — > T is a Nash embedding if (j){S) is a Nash submanifold of T and the 
restricted map (/> : 5" — > (f){S) is a Nash isomorphism. 

In the natural way, we define Nash Lie groups, Nash group actions, etc. 

We can define in the obvious way Nash fibrations and Nash coverings of Nash manifolds. 
We note that local triviality in the etale topology on real algebraic varieties implies local 
triviality in the Euclidean topology. 

Then in particular we get the notion of a Nash vector bundle over a Nash manifold 
and the space of Nash sections. If X is a Nash manifold then the tangent and cotangent 
bundles of X have natural Nash bundle structures. Consequently, for any tensor field r] on 
X, such as a vector field, a differential form, a metric or a complex structure, we define rj 
to be Nash if the corresponding section of the bundle TX®"^ T*X^^ is Nash. This gives 
notions of Nash symplectic manifold, Nash Riemannian manifold, Nash complex manifold, 
Nash Kaehler manifold, Nash hyperkaehler manifold, etc. 

If X is a totally real Nash submanifold of a smooth complex algebraic variety Z such that 
dims X = dime Z, then we say that Z is a Nash complexification of X. A stronger condition 
on X is that X is a Euclidean connected component of the fixed-point set Z'^ for some real 
structure k on Z. Then we say also that X is a real form of Z. This extends our definition 
of real form from § A.5| . 



A. 10 Orbits of Real Algebraic Groups 

We consider now real algebraic groups G(M) that arise in the following way. Let G be a 
Zariski connected complex algebraic group defined over R with group G(M) of real points. 
We assume as usual that G is a complex affine algebraic variety; then G(M) is a real affine 
algebraic variety. For example, compact Lie groups arise in this way. 

Now G(M) is Zariski connected but in general not Euclidean connected. For instance if 
G = GL{n,C), n > 1, then G{M) = GL(n,M) has two connected components, defined by the 
sign of the determinant. The Euclidean connected component Gk of G(M) is a semi- algebraic 
set in G(M). If G is semisimple and simply-connected, then Gr = G(M). 

Suppose G acts morphically on an (irreducible) complex algebraic variety X, i.e., G acts 
on X and the action map G x X — > X is a morphism of complex algebraic varieties. If X 
and the action (i.e., the action morphism) are defined over M then G(M) acts morphically on 
the (irreducible) set X(M) of real points. Each Euclidean connected component Xjr of X(M) 
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is a semi-algcbraic set in X(IR). 

Each G-orbit G • x on X is a smooth locally closed (irreducible) complex algebraic sub- 
variety of X. Hence, if x G -^(M), the set of real points 

{G-x){R) = {G-x)nX{R) 

is a smooth locally closed (irreducible) real algebraic subvariety of X(M), and hence is a 
finite union of Euclidean connected components of the same dimension. These components 
are then semi-algebraic sets and moreover are Nash submanifolds. 

On the other hand, by the Tarski-Seidenberg Theorem, the orbits G(M) • x and ■ x are 
semi-algebraic sets in X(M). In particular Gr ■ x is a component of (G ■ x)(M). 

Thus Nash manifolds are the natural objects in this setting. Finally we give an example 
of how Nash isomorphisms can arise. Consider the standard action of G = 5*0(3, C) on as 
the special orthogonal group of the quadratic form x^ + y'^ — where x,y,z are real linear 
coordinates. The subset 

X = {x^ + y^ = z^)-{{Q,Q,())} 

is a G-orbit. 

But X(M) has two Euclidean connected components defined by the sign of z. Let X-g^ be 
the component where z > 0. The projection 

p : X(M) ^ C - {0}, p{x,y,z) = x + iy 

is a 2-to-l etale real algebraic morphism. The restricted map : X^ — > C — {0} defined by 
p is a Nash isomorphism. Indeed the inverse map is 

C - {0} ^ Xr, x + iyt-^ (x, y, y/^Ty^^ 
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